The way I see it is that the lever is set to the position it is in for a reason, ie trains are expected to run down that particular line, so I wouldn't switch it.
Switching the lever might send the train onto a track going the wrong direction - and cause a head-on collision with another train coming the opposite direction for example.
If there were that many people standing on the two tracks, could it be that they are actually employees working on the lines? - in which case the single person would not be expecting a train to appear, but the other three would be - regardless of the arrival time. When track workers are on 'active' lines, in the UK at least, they place detonators further up the line which make a sufficiently audible noise when an approaching train triggers them - thus alerting the workers to get off the line in plenty of time.
Even if they aren't track workers - the train should have a loud enough hooter (what the hell do you call those things LOL!) to warn everyone to get out of the way.